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COMPUTING  STATIONARY  POINTS,  AGAIN 


B.  Curtis  Eaves 


1.  Introduction  and  Abstract 


kGiven  a nonempty  set 


~tx  > <y  : Ax— *•-*}" 


with  an  affine  function^  $ (x)  - Cx  *— c — fraa  °K  -to  we  con- 

sider (existence  and)  computation,  in  a finite  number  of  steps,  of 

★ ^ 

a stationary  point. \ A point  x in  t.  is  defined  to  be  a 

stationary  point  o %)  (^( , s>  if  x • £(x  ) 2.  x ' lp(x  ) for  x 

in  ^The  existence  and  computation  of  stationary  points  is,  in 
particular,  central  to  the  solution  of  certain  quadratic  programs, 
matrix  games,  and  economic  equilibrium  problems.  Computing  a 


stationary  point  of — (-= 
complementary  problem  , 


is  equivalent  to  solving  the  linear 


x > 0 X > 0 V > 0 s ^ 0 
x • p « X • s ■ 0 


in  the  sense  that  if  x is  a stationary  point,  then  there  is  an  5 

(x  , X,  p,  s)  solving  (1),  and  reversely,  if  (x,  X,  p , s)  solves  sw:iWI 


(1) , then  x is  a stationary  point. 


JUSTIFICATION 
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Towards  describing  oyr  result,  let  x be  any  extreme  point 
of  . Consequently,  there  is  a submatrix  (L,  i)  of  n rows  of 
the  matrix. 


such  that  Lx  * i and  L is  nonsingular.  For  any  such  L let 
p * L f where  |>  is  any  positive  vector.  We  shall  use  the  family 
of  stationary  problems  where  £g(x)  4 £(x)  + ®P  and 

0 > 0 in  order  to  solve  the  stationary  problem  (*£  , £Q)  = (T(,  £) 
Observe  that  x^  is  a stationary  point  of  J?e>  for  all 

sufficiently  large  6;  let  0^  be  the  smallest  such  0. 

Define  a piecewise  linear  path  to  be  a function  from  [0,  + «) 
to  R that  Is  affine  on  each  element  of  a finite  closed  cover  of 
[0,  + 0D) . The  following  theorem  captures  the  principal  result. 


Theorem:  Lemke’s  algorithm  computes  a piecewise  linear  path  (X,  0) 
such  that  X(0)  = x^,  0(0)  * 6®,  X(t)  is  a stationary  point  of 

, £q(c))  f°r  c»  an<*  either  0(t)  » 0 for  some  t or  X(t) 
tends  to  infinity.  S 

In  our  previous  study  [2]  a similar  result  was  developed,  however, 
there  the  algorithm  operated  by  perturbing  rather  than  £ . There  1C 
was  of  form  {x  : Ax  £ a},  was  not  required  to  have  an  extreme  point,  and 
the  path  could  be  initiated  anywhere  in  ^ . Nevertheless,  the  present 
scheme  should  be  more  effective  if  applicable,  namely,  if  % lies  in  the 
nonnegative  orthant  and  the  algorithm  is  to  be  initiated  at  an  extreme  point. 


* 


Although  Mylander  [4;  Ch  A]  was  studying  quadratic  programming 
(C  was  assumed  symetric)  under  a nondegeneracy  condition  and  from  a 
perspective  of  parametrizing  the  feasible  region  ( [°f-  was  deformed  via 

{e  • x <_  0})  rather  than  the  objective,  much,  or  perhaps  all,  of  the 
Theorem  can  be  extracted  from  his  proof.  In  [1;  § 11]  it  is  shown  that 
Lemke's  algorithm  can  be  used  to  compute  (x,  x,  0,  0)  so  that  x + tx 
is  a stationary  point  of  , £(0  + t 0))  for  all  t >_  0 where  0-0 
or  x f 0;  here  both  ^ and  were  parametrized. 


2.  A Special  Case 

In  this  section  we  assume  is  the  origin  and  (L,  O = (I,  0) , 

and  under  these  conditions  we  prove  the  Theorem.  Let  p in 
Rn  be  any  positive  vector  and  we  apply  Lemke’s  algorithm  (see  [1]  or 
[3])  to  the  augmented  linear  complementary  problem 


c :’)  o * o * o 


x > 0,  X > 0.  p > 0,  s ^ 0,  0^0 


x • p « X • s - 0 


Conceptually  the  first  step  of  the  algorithm  is  to  perturb  c and  a 


to  c^.  ■ c + (e,  e11)  and 


, n+1  n+m. 

a + (e  » • • • » t ) , 


respectively,  where  e is  a positive  infinltesmal;  we  obtain  the  per- 
turbed augmented  linear  complementary  problem: 


(2,0 


(.: :’)  o * (::)•  o 


x^o,  x^o,  p > o,  s > o,  e>_o 


x • p ■ X • s * 0 


Clearly  (x,  X,  p,  s,  0)  - (0,  0,  c^  + 0p,  a£  0)  for 

all  sufficiently  large  0 forms  a ray  of  solutions  to  (2,0;  this 
ray  is  referred  to  as  the  primary  ray. 


I 


The  initial  solution  (x®,  X^,  y^,  s®,  0^)  is  generated  by 


selecting  the  element  of  the  primary  ray  with  the  smallest  possible  0. 


Henceforth  we  assume  that  0^  is  positive  for  if  0^  • 0,  the  path  of  the 


Theorem  is  trivially  obtained  by  setting  (X(t),  0(t))  - (0,  0)  for  all  t. 


Beginning  with  the  initial  solution,  Lemke's  algorithm  complementary 


pivots  and  generates  a sequence  of  solutions  (x*,  X*,  y*,  s*,  0*) 

£ £ £ € £ 


i = 1,  ....  k for  (2,e).  Each  component  of  each  solution  is  a poly- 
n+m  k 

nomial  of  form  £ f c . The  algorithm  terminates  if  0 * 0 or  if 

ft  1 £ 


a ray  is  encountered.  Such  a ray  is  referred  to  as  the  secondary  ray 


and  has  at  most  one  point  in  common  with  the  primary  ray. 


By  x*i  we  denote  the  quantity  xC  where  c has  been  set  to 

w £ 


zero,  etc.  First  let  us  define  the  function  (X,  A,  M,  S,  0)  on 


[0,  k]  by  setting  (X,  A,  M,  S,  0)  (t)  ■=  (xj,  xj,  yj,  sj,  ej)  for 


t = 0,  1,  ...,  k and  by  extending  it  affinely  on  [i,  i + 1]  for 


i “ 0,  . . . , k - 1. 


If  the  algorithm  terminates  with  0^-0  we  obtain  a path  of 


solutions  to  (2)  with  0(k)  * 0 by  extending  (X,  A,  M,  S,  0)  con- 


stantly on  (k,  + °°) , and  we  obtain,  in  particular,  the  path  (X,  0) 


of  the  Theorem. 


So  now  let  us  assume  the  algorithm  terminates  on  a secondary 


ray.  This  ray  of  solutions  to  (2,e)  will  have  the  form 


, k , k k k „k.  . ^ - - - - 

(*  K’  » . eJ  + c (*.  x*  V,  s,  0) 


-----  k k - - 

with  t >_  0 where  (x,  X,  y,  s,  0)  i 0,  (x£,  X£,  x,  X)  + 0,  and 


c :)  o *o  ■ • Q 


li 


■ ft  I 

W 


Our  task  is  now  to  show  that  x j*  0.  Suppose  x = 0 and 

Ic  lc  “ 

x£  “ 0,  then  s£  > 0,  then,  using  X • s * 0,  we  get  X - 0 and 

X * 0 which  contradicts  a property  of  a secondary  ray.  So  now 

_ k — 

let  us  suppose  that  x = 0 and  x^  j4  0.  We  have  8*0, 

- . k , r k T , k - k - k.Tr-  k - 

X • Ax  + X • s =X*a,  and  x • p - x • Cx  - x • A X * x • p0 . 

E E E E E E E 

Combining  the  last  two  expressions,  using  p • x - 0,  s • X * 0,  and 

k k 

x * 0 we  have  X • a * x * x • p0  . Hence,  X = 0 and  0*0  or 

E E E 

(x,  X,  p,  s,  0)  = 0 which  is  again  a contradiction.  We  may  conclude 
that  x i*  0.  A path  of  solutions  to  (2)  with  X(t)  tending  to  infinity 
is  obtained  by  extending  (X,  A,  M,  S,  0)  to  [0,  + ®)  by  setting 


(X,  A,  M,  S,  0)(t) 


, k k k k „k. 

<x0.  Ao*  u0’  s0*  0O) 


+ (t  - k) (x,  X,  p,  s,  0) 


for  t >_  k.  In  particular,  (X,  0)  is  the  path  of  the  Theorem. 
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3.  Overview  of  Transformation 

We  shall  prove  the  Theorem  by  transforming  the  general  case  into 
the  special  case  of  the  previous  section.  Here  we  give  an  overview  of 
the  transformation  used. 

Consider  the  stationarity  problem  (*•)£,  g ) in  Rn  and  let 
^(x)  * Lx  + t be  a one  to  one  affine  transformation  from  Rn  to  Rn. 

Let  and  /£  (x)  ■ Cx  + c where  CU  1T  CL-1, 

- -if 

c 4 L c - CL,  and  T denotes  transpose. 

Lemma  1:  x*  is  a stationary  point  of  (%,£)•  if  and  only  if  x*  = of(x*) 
is  a stationary  point  of  (*.£) 

* * * 

Proof:  x • (Cx  + c)  ^ x • (Cx  + c)  for  all  x in  if  and  only  if 
(L_1 (x  - *))  • (C(L~1(i*  - £))  + c)  > L_1(i*  - £))  • (C(L_1(x*  - l))  + c 
for  all  x in  % S 

Note  that  the  lemma  does  not  require  any  assumptions  on  , the 

subset  of  Rn.  An  interesting  case  of  Lemma  1 is  obtained  when  L is 

-IT 

orthogonal;  namely,  when  L * L. 


I 


*>  «V. 
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4.  The  General  Case 

Here  we  prove  the  Theorem  for  general  £)  by  using  a 

transformation  as  described  in  Section  3 to  obtain  the  special  case  of 
Section  2. 

For  a matrix  B we  denote  by  B and  B the  submatrix 

*Y  Y* 

of  rows  and  columns,  respectively,  indexed  by  y;  if  B is  a vector 

we  drop  the  dot.  Let  B be  a subset  of  {1,  ....  m + n}  of  size  m 

and  let  a be  the  complement  of  B where  A is  m x n. 

Given  the  extreme  point  x^  of  °X.  let  s^  * a - Ax^.  We 

say  that  B is  a basis  index  yielding  (x^,  s^)  if  (A,  I)  has  an 

• B 

inverse  and 


(3.~  C\ 


(A,  I >:J  a 


Observe  that  B is  a basis  index  yielding  (x^,  s^)  if  and  only  if 


(.: ) ■ t: ). 


UL 


and  is  n°nsingular.  Using  the  simplex  method  if  necessary, 

a* 

select  any  such  a and  B. 

Define  the  map  by  «x) 

where 


Lx  + l 


L £ 


C L “■* 1 4C 1 
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Clearly  Is  a one  to  one  affine  map  from  ^ to  °)t  where 


*X  A L 0 : Ax  _<  a} 


A 4 (A,  1)7 J (A,  I).o 


a A (A,  I)eg  a 


It  is  important  to  note  that  a ^ 0 and  «^(x^)  * 0.  We  form  the 
stationary  problem  (<y,]|)  by  setting 


Jft(x)  4 Cx  + c 

-IT  -1 
C 4 L CL 

-IT 

c 4 L c - Ci 


Let  p be  any  positive  vector  and  set  p 4 L p.  Define 

jg0(x)  A J200  + ®P  and  g (x)  4 g(x)  + 0p.  Of  course,  Lemma  1 

applies  to  the  two  systems  (*)(.,  {p 0)  and  (**U  ; furthermore, 

a > 0 and  x^  4 <*(x°  ) * 0,  hence,  the  special  case  of  Section  2 

applies  to  with  the  extreme  point  x^  = 0. 

Now  consider  the  analogue  of  (2)  for  (*%,  ) , namely 


(IDO  •(;)•(!)•■(!) 


x>^0  X > 0 p > 0 s ^ 0 


x • p « 0 X • s ■ 0 


For  this  particular  type  transformation  Lemma  2 strengthens 
Lemma  1. 

Lemma_2_:  (x.  A,  p,  s,  0)  solves  (2),  if  and  only  if  (x,  A,  p,  s,  0) 
solves  (2)  where 


Proof:  We  represent  (2)  with  the  schema 


x>0  A > 0 p :>  0 s > 0 0 > 0 


x • p = A • s ' 0 


After  regrouping  the  variables  we  get  the  next  system  which  has 
the  same  solutions  (x.  A,  p,  s,  6). 


■WPMMMMMMHf 


5.  An  Example 

Consider  the  stationarity  problem  (^,£)  defined  by 


The  region  is  shown  in  Figure  1. 
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Assuming  xu  • (0,  2)  we  get  o ■ {1,  3},  8 ■ {2,  A,  5), 
p - LT  p - (2,  1)  and  (L,  O - J _fj  where  p - (1,  1).  The 

system  (2)  can  be  expressed  as  (A) . 


I 


y2  ®1  S2  s3 


0 0 0 0 


B 


The  algorithm  can  be  executed  by  pivoting  sequentially  on  this  matrix  of 
(A)  at  positions  (2,  3),  (3,  2),  (2,  11),  (5,  8),  (1,  5),  (1,  1),  (A,  10) 
(A,  A),  and  (2,  5).  The  path  (X,  0)  generated  is  defined  by 


and  X is  displayed  in  Figure  2. 


X(3) 
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6.  Appendix 

This  section  demonstrates  that  the  involvement  of  a degeneracy 
discussion  in  Section  2 was  necessary  in  order  to  cover  the  possibility 
that  it  might  contain  some  zeros. 

Consider  the  application  of  Lemke's  algorithm  to  the  augmented 
linear  complementary  problem. 

Bz  + q + p0  * w 

z _>  0 w ^ 0 6^0  z • w ■ 0 

In  order  to  initiate  the  algorithm  one  needs  (p,  q)  lexico  non- 
negative row  by  row.  However,  if  (p,  q)  is  merely  lexico  non- 
negative and  not  lexico  positive,  then  Lemke's  algorithm  may  terminate 
on  a ray  that  is  Identical  to  the  primary  ray  after  the  perturbation 
is  dropped.  To  illustrate  this  point,  consider  the  data 

-r.)  -o 

2 

and  perturb  q to  q£  ■ (-1  + t,  e ). 

Lemke's  algorithm  begins  with  the  solution  (w,  z,  0) 

2 

- (0,  e , 0,  0,  1 - e),  iterates  through  the  solutions 

2 2 

(0,  0,  e , 0,  1 - e)  and  (0,  0,  0,  e , 1 - e),  and  terminates  with 
the  ray  of  solutions  (0,  0,  0,  e2,  1 - e)  + t(l,  0,  0,  0,  1)  with 
t ^ 0.  Upon  dropping  the  perturbation  we  see  that  the  secondary  ray 
of  the  perturbed  problem  becomes  the  primary  ray  of  the  original  problem. 

18 
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